|
Eager Space | Videos | All Video Text | Support | Community | About |
|---|
Mia and the martians:
https://www.indiegogo.com/projects/mia-and-the-martians-publishing-campaign#/
Off Nominal Podcase episode
https://offnom.com/episodes/166
Rocket Factory Augsburg test anomaly update: https://www.youtube.com/watch?v=ORviDaDsj7U
Rockets Behaving Badly - Pogo
https://www.youtube.com/watch?v=Fn9hAnaoDfE
I've talked about raptors and how their reliability will affect the reliability of Starship overall, but now that we have 3 successful launches there's some useful data and I've come up with what I hope is a better way of explaining things.
I need to start with a few caveats.
To evaluate reliability, NASA commonly uses an approach called probabilistic risk assessment. In this approach you look at every component in your system and predict how likely it is to fail, and then combine those together into large components, and finally into your whole vehicle, factoring in the redundancy that you have built into your system. That spits out a number.
If you have heard that the probability of loss of crew (LOC) for crew dragon is 1 in 276, that number comes from this sort of assessment.
But it's not a real number, which I can illustrate with three examples.
On June 28th, 2015, the second stage of Falcon 9 exploded during the CRS-7 mission. On September 1st, 2016, a Falcon 9 exploded during fueling before a static fire on the AMOS-6 mission. On April 20th, 2019, a crew dragon capsule exploded during routine testing of the super draco abort engines.
What caused these failures?
On CRS-7, a strut holding a helium tank was understrength and broke, letting the tank break free and overpressurizing the second stage.
On AMOS-6, a composite overwrapped pressure vessel exploded due to the creation of solid oxygen in the composite weave.
On the crew dragon test, the interaction of a titanium valve with a hypergolic propellant led to an explosion.
The commonality between all these failures is that they don't show up in a PRA assessment. "Strut weaker than the specification" didn't show up in the possible failure modes, and both of the AMOS-6 and crew dragon explosions were failure modes that had never been seen before. "Blew up because of something weird nobody had seen before" is also not in the list of failure modes.
That's the big weakness of PRA - you need to be able to conceive of a failure mode and assess how likely it is to properly factor it into the assessment. And that should make you skeptical of the numbers out of an analysis like PRA.
The alternative is to use empirical data. Test the hell out of all the individual parts and fly all the time, and you'll get a decent idea how reliable your vehicle actually is.
The problem with that most rockets don't fly very often. Delta IV flew using the RS-68 a total of 45 times in the medium and heavy variants with only one partial failure. So how reliable is it?
The RS-68 could be a 1 in 25 engine and we just got lucky. Or it could be a 1 in 250 engine, or even better than that. We just don't know.
There are only two rockets with decent empirical numbers. Soyuz has launched 1680 times in various forms, so we have a lot of information on the RD-107 and RD-108 engines that it uses. And Falcon 9 has 389 launches.
And just to complicate things a bit more, both the Soyuz and Falcon 9 engines have been modified along the way.
Returning to Raptor, we have some real empirical data to play with. IFT2 and IFT-3 both had all engines light and work until staging, and IFT-4 had one engine fail to light but the remaining 32 of 33 worked until staging.
So that's 98 out of 99, which I'm going to round up to a demonstrated 0.99 reliability. There are significant caveats on that number - we could be lucky or unlucky - but let's just assume it's the actual number for sake of argument.
If we have an engine like that, what would we expect in terms of reliability?
Let's say we're talking about the RS-68 engine that flew on the Delta IV series, and let's assume it has 0.99 reliability.
If we're flying the delta IV medium with a single engine, we don't need to do any math - 99 times out of 100 it will be successful...
But what if we are flying the delta IV heavy with 3 RS-68 engines. The chance of success is 0.99 but we have three engines, so we multiply those probabilities together - which is the same as raising the base probability to the power of the number of engines - and we find that the overall probability of success is 0.97.
That's about a 1 in 33 chance of failure, which makes sense.
There's an underlying assumption that shutdown equals failure, and that's not quite true.
During the launch of Apollo 13, the center engine of S-II second stage shut down two minutes early because of chamber pressure fluctuations.
It was, however, about two thirds of the way through the stages burn and the other engines were able to run a little longer, and it did not lead to a loss of mission.
See my video on POGO for more information.
Similarly, STS-51F had a main engine fail three and a half minutes into the flight, but the remaining engines were barely able to put the shuttle into orbit, though the orbit was lower than planned.
There is math to try to factor when an engine might shut down during launch into reliability calculations, but I'm not going to get into it here, and I'll assume that losing an engine always happens early enough in a launch for it to be problematic.
Time for a graph.
If our engine is 0.99 reliable, then we can expect that it will fail once out of every 100 uses. That's pretty straightforward.
As we add engines, the reliability goes down, and by the time we get to 8 engines, we can expect an engine failure every 13 flights.
Having a bunch of engines is clearly a bad idea.
And then we add a ninth engine, and something unexpected happens. If we use our equation of raising the reliability of 0.99 to the power of the number of engines. we should get a value of 1 in 12, way down here at the bottom of the graph.
But when we get to 9 engines, a magical thing happens, and that magical thing is engine redundancy. We have enough engines now that if one engine fails, the remaining engines are powerful enough that they can still complete the mission. Our booster is "one engine redundant"
Our chance of one engine failure is an alarming 0.086, but for the booster to fail we need two engine failures, and the probability of that is 0.0075, or 1 in 134.
That's better than our original reliability, but it's not a lot better. Hold onto that thought, I'll be back to it in a bit.
You may be wondering why the magic number is 9. I chose 9 because we know that Falcon 9 is single engine redundant
8 probably works and 7 might work, but fewer than that gets less likely - we are nearing the Saturn V example and that rocket isn't getting off the pad with only 4 working engines.
Back to our question - dealing with the hassle of 9 engines seems like a lot of work if you only get a 30% increase in reliability.
I chose a rocket engine that is only 99 percent reliable, and that's not a great engine. What if we can do better?
This is an engine that has 0.995 reliability, or 1 in 200. With 9 engines, that's 1 in 513, or 2.6 times better than a single engine. That is getting interesting...
Bumping up to 1 in 500, or 0.998, the reliability with 9 engines goes up to 1 in 3136, or 6.3 times the reliability of one engine.
I'm sure you don't expect me to stop here....
If you can built an engine with what we would call "three 9s" of reliability - with 1 in 1000 chance of failure, our 9 engine booster is expected to have a reliability of 1 in 12,445, or 12.4 times the reliability of 1 engine. That's a full order of magnitude better than the single engine solution.
The question is whether we should actually care about this.
The mighty F-1 engine that powered the Saturn V flew 13 times, so it only had 65 engine flights. If it's a 1 in 200 engine - and it probably is at least that good - the chances of an in-flight failure are small.
409 RS-25 engines flew on the shuttle and now SLS. Empirically, it's about a 1 in 400 engine.
Falcon 9 has flown 389 total launches, so that's 3501 Merlin 1D flights. And at those sorts of flight volumes, I care quite a bit about the difference between 1 in 1000 and 1 in 12,000. My analysis is that the Merlin 1D is a 1 in 1000 engine, perhaps a bit higher.
We are already exposed to the value of engine redundancy.
Normal FAA rules require that any two-engine airliner always be within 60 minutes of an airport, so they can reach a safe place within an hour if one engine fails. We therefore map out 60 minute circles and choose a flight path where we are always inside one of these circles.
That's unfortunately inefficient because we aren't flying the most direct route.
The FAA has therefore created a program called extended-range twin-engine operational performance standards, or ETOPs
If your plan and engines are good enough - if they have a higher reliability - the FAA will let you extend the 60 minutes to a larger value.
If a plane is approved for ETOPS 120 - like the airbus 320 - then it can fly the shortest route from new York to London as it is always withing 120 minutes of one of those airports.
For flights in the Pacific, this becomes more important because the distances are so much longer.
Airplanes that fly these routes might be ETOPS 180, 240, or even 330 minutes like this Boeing 777.
It's all based on engine redundancy - twin engine airliners must be able to fly with only one engine.
But you came here to hear about raptor and starship.
I'm going to assume that every 9th engine gives you redundancy. It might be a different number but it's not going to change the results much.
Here is super heavy at 0.99 reliability. We see the same pattern as before - every 9th engine gives us a nice bump in reliability - and then engines added after that reduce the reliability.
27 engines not surprisingly gives us the best result, at about 3.1 times the reliability, but adding 6 more engines bumps it down to only 1.6 times the reliability. Not very exciting for a booster that has 3 engine redundancy, where it takes 4 engines to fail to have real issues.
It's pretty unlikely that Raptor is a 1 in 100 engine. Let's explore if it is better.
If Raptor is only 1 in 200, 27 engines is 1 in 3900, and 33 engines drops down to 1 in 1850. Only 9 times the reliability of 1 engine.
But Raptor's probably not a 1 in 200 engine....
At 1 in 500 or 0.998 reliability, it starts to get silly.
27 engines gives us a 1 in 130,000 chance of failure, and going up to 33 engines drops that down to 1 in 60,000
Time to take the final step...
If raptor is a 1 in 1000 engine - if it's as good as merlin 1d - 27 engines gives you pretty close to a 1 in 2,000,000 million chance of a failure of the stage, and 33 engines drops that down to only 1 in 900,000. It is *900* times better than a single engine solution, or pretty close to 3 orders of magnitude.
At this point, I'm sure some of you have questions. You, the one in the back waving your hand around?
Decaying Flavors? Huh?
Oh, Cascading Failures.
I guess that means I have to show the video.
On August 9th of 2024, Rocket Factory Augsburg was performing a static fire and they had an anomaly that led to cascading failures when the stage failed to shut down. The analysis video is linked in the description is worth a watch.
I wanted to show this because a) it's fun to watch and b) to emphasize that this sort of thing doesn't really happen in practice. I've watched a *lot* of rocket explosions videos over the years and have never seen this sort of failure before.
Rocket engines are designed to shut down rather than blow up and take the rest of the vehicle with it.
But it's certainly true that this risk does exist in rockets with multiple engines, but there is a mitigation...
Let's say you are ULA and you are launching a Vulcan, which is powered by 2 BE-4 engines.
One of those engines starts acting hinky. Do you shut it down?
If you do, your payload will end up in the ocean.
If you let it keep running, maybe it works long enough to finish the mission.
So you set your engine shutdown parameters a little loose because the worst thing you can do is shut it down too early.
If you are flying astronauts and they have an abort system, you probably approach it a bit differently as they are safer in an abort with a shut down than one with an exploding rocket stage, but there are some cases where that isn't true.
I talked earlier about the STS-51F shuttle mission where an RS-25 engine shut down, and after that happened the crew disabled automatic shutdown on one of the other engines that was starting to act hinky, and that saved the mission.
If you have engine redundancy, the decision tree is different. If an engine starts acting hinky, you shut it down because doing so does not compromise engine success and if you keep it running it might break other engines.
This is especially true if your vehicle is reusable. Shut the engine down before it gets damaged and the fix is going to be simpler and cheaper than if you let it run longer.
It's therefore possible that the engine that didn't light for IFT-4 didn't actually have a problem but just looked a little hinky, and since they don't actually need all 33 engines running it automatically shut down.
For super heavy, it's not going to the raptors that are the problem, it's going to be everything else that is required to get through the missions successfully. Far more likely to have a problem elsewhere.
We can also spend a little time looking at starship.
Starship has 3 small nozzle sea level engines in the middle and 3 large nozzle vacuum engines on the outside.
Is this arrangement engine-redundant? It's not clear; SpaceX hasn't talked about this and the requirements for second stages are easier than first stages because they have smaller gravity losses.
Starship 3 is planned to have 3 sea level engines and 6 vacuum engines around the outside, as shown in the photoshopped image.
There will be engine redundancy with this arrangement.
Starship needs a single functional sea-level engine to land successfully.
That means for it fail, 3 engines need to fail. If they are 1 in 100 in this scenario, we need to hit the "1" part three times, so it's a 1 in 1,000,000 chance.
And that's with only a 1 in 100 reliability.
Like super heavy, we should worry about other things than engine reliability for starship landings.
And that's the story...
Multi engine redundancy will make the chance of normal raptor failures a non-issue on super heavy and we can expect high reliability on Starship 3 for both launch and landing.
If you liked this video, maybe you would enjoy a concert.
But actually, what I'd really like you to do is go to the Mia and the Martians project on indiegogo and pledge your support.
What this world needs is more cool space books for kids.
Link is in the description, along with a link to the Off Nominal podcast discussion of the book.